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Office Action Summary 



Application No. 

09/718,374 



Examiner 

Seyed Azarian 



Applicant(s) 

LIANG ET AL. 



Art Unit 

2625 



The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )K Responsive to communication(s) filed on 29 January 2003 . 
2a)K This action is FINAL. 2b)n This action is non-final. 

3) \3 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) n Clalm(s) 1-38 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) 6-9,30 and 36-38 is/are allowed. 

6) n Claim(s) 1-5, 10-29 and 31-35 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) 13 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on Is/are: a)n accepted or 5)0 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
11 )D The proposed drawing correction filed on 24 November 2002 is: a)K approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)nAII b)n Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. , 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 

1) □ Notice of References Cited (PTO-892) 4) □ Inten/lew Summary (PTO-413) Paper No(s). . 

2) □ Notice of Draftsperson's Patent Drawing Review {PTO-948) 5) [H Notice of Infomrial Patent Application (PTO-152) 

3) □ Infomnation Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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FINAL ACTION 



RESPONSE TO AMENDMENT 



1 . Applicant's amendment filed, January 29, 2003, has been entered and made of record. 

2. Applicants' arguments with regards to Claims 1-5, 10-29 and 31-35, have been fully 
considered but they are not persuasive. 

Applicant argues in essence that Maki does not teach "characterize activity of said object 
of interest based on changes in position and shape over time such as eating, running and 
sleeping". With respect to applicant's argument the Examiner disagrees and indicates refer to 
column 24, line 63 through column 25, linel 1, described the motion of the object automatically 
or acquire information on the "position of the object or on the change of the posture", on the 
basis of the time-series images, and posture information detector 303, tracks the motion of the 
object automatically according to the position of the feature points. 

Furthermore column 32, lines 56-67, refer to target object and configured to determine 
pixels corresponding, and position and posture of target, and finally column 27, lines 20-31, refer 
to comparison section then evaluates the various postures according to the similarity of the 
generated images to the image of the target object and estimates the posture on the basis of the 
evaluation. 
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Claim Rejections - 35 U.S.C, §103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-5, 10-29 and 31-35, are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Maki et al (U.S.6,072,903) in view of Hsieh (U.S.5,546,439). 

Regarding claim 1, Maki et al discloses a system, comprising a computer configured to 
determine a position and shape of an object of interest from video images and characterize 
activity of said object of interest based on analysis of changes in said position and said shape 
over time, wherein said object of interest includes eating jumping, rurming, (see column 15, lines 
49-57, refer to detecting information about position, shape and posture of object, also column 32, 
lines 56-67, refer to target object and configured to determine pixels corresponding, and position 
and posture of target column 24, line 63 through column 25, linel 1, described the motion of the 
object automatically or acquire information on the "position of the object or on the change of the 
posture", on the basis of the time-series images, and posture information detector 303, tracks the 
motion of the object automatically according to the position of the feature points). 
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However Maki et al does not explicitly state " computer configured " . On the other 
hand Hsieh, teaches (column 6, lines 58-67, the "computer configured to identify, for an object 
of interest"). 

Therefore it would have been obvious to a person of ordinary skill in the art at time the 
invention was made, to modify Maki et al invention according to the teachings of Hsieh because 
it is processing part of computer or information detecting unit that enable to identify object of 
interest which fully satisfies the requirements of claim. 

Regarding claim 2, Maki et al discloses the system of claim 1, further comprising: a 
video camera coupled to said computer for providing said video images, (see column 23, line 64 
throw column 24, line 7, tracking of motion picture, by use of television camera). 

Regarding claim 3, Maki et al disclose the system of claim 2, further comprising: a video 
digitization unit couple to said video camera and said computer for converting said video images 
provided by said video camera fi-om analog to digital format, (see above claims and column 8, 
lines 46-51, refer to using a digital filter). 

Regarding claim 4, Maki et al discloses the system of claim 3, further comprising: a 
storage/retrieval unit coupled to said video digitization unit, said video camera, and said 
computer, for storing said video images and standard object video images, (see column 25, lines 
32-37, refer to storage section). 

Regarding claim 5, Maki et al discloses the system of claim 1, wherein said computer 
includes an object identification and segregation module receiving said video images, (see 
column 29, lines 23-28, refer to receiving three-dimensional position information). 
Regarding claim 13, Maki et al disclose the system of claim 1, wherein said object is a mouse, 
(see column 2, lines 20-26 refer to three dimensional mouse). 

Regarding claim 16, Maki et al discloses a method of determining and characterizing 
activity of an object using computer processing and identifying and classifying the changes, (see 
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column 3, lines 15-22, refer to changes of image in different time, and also column 27, lines 20- 
31, refer to comparison and synthesizing the images). 

Regarding claim 19, Maki et al discloses the method of claim 18, wherein said step of 
generating a background image includes the step of determining variation in intensity of pixels 
within said individual frames to identify a region where said foreground object is located, (see 
column 19, lines 44-52, corresponding to pixel and intensity of the light). 

Regarding claim 26, Maki et al discloses the method of claim 24, wherein said step of 
identifying and classifying changes to said foreground object uses contour-based shape 
information selected from the group consisting of b-spline representation, convex hull 
representation, and comer point, (see column 9, lines 14-23, refer to connected feature points, 
such as outer comer of the eye ). 

Regarding claim 27, Maki et al discloses the method of claim 24, wherein said step of 
identifying and classifying changes to said foreground object includes identifying a set of model 
postures and their description information, said set of model postures including horizontal 
posture, vertical posture, eating posture, or sleeping posture, (see column 25, lines 6-11, refer to 
posture information). 

With regard to claims 10-15, 17-18, 20-25, 28-29 and 31-33, the arguments analogous to 
those presented for claims above are applicable. 

5. Claims 34-35, are rejected under 35 U.S.C. 103(a) as being unpatentable over as applied 
to claim above, and further in view of Smith et al (U.S.5,870,138). 

Regarding claim 34, Maki et al discloses the method of claim 33, wherein said temporal 
analysis is a time-series analysis such as Hidden Markov Model (HMMs), (see above claims). 

However Maki et al and Hsieh does not explicitly state " Hidden Markov Model " . On 
the other hand Smith et al teaches (column 17, lines 41-43, the device can be implemented as a 
Hidden Markov Model and has two phases of operation, namely training and recall). 
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Therefore it would have been obvious to a person of ordinary skill in the art at time the 
invention was made, to modify Maki et al and Hsieh invention according to the teachings of 
Smith et al because it is device to map the input data (images) from the camera to outputs which 
represents the probability of the input images belonging to a specified set of expressions, and can 
easily be implements in an images device such as video camera. 

With regard to claim 35, the arguments analogous to those presented for claim 34, is 
apphcable. 

Allowable claims 

6. The following is an examiner's statement of reasons for allowance. 

The instant invention recites method and system for providing and /or activities of an 
object using video. Claim 6, representing claims 7-9, 30, and 36-38, are allowable due to the 
Performing a computer configured to determine a position and shape of an object of interest from 
video images and characterize activity of said object of interest based on analysis of changes in 
said position and said shape over time, wherein said computer includes an object identification 
and segregation module receiving said video images, and wherein said object identification and 
segregation module receiving said video images and wherein object identification and 
segregation module operates using a background subtraction algorithm in which a plurality of 
said video image are grouped into a set, a standard deviation map of the set of video images is 
created, a bounding box where a variation is greater than a predetermined threshold is remove 
from said set of video images and set of images less said bounding boxes is averaged to produce 
a back ground image. 

These key features in combination with the other features of the claimed invention are 



• 
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Neither taught nor suggested by the art of record. 



Thus, claims 6-9, 30 and 36-38, are allowed. 



Conclusion 



8. THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of 
the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 

MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 

MONTHS of the mailing date of this final action and the advisory action is not mailed until after 

the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 

will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 

CFR 1.136(a) will be calculated from the mailing date of the advisory action, hi no event, 

however, will the statutory period for reply expire later than SIX MONTHS from the date of this 

final action. 



9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Seyed Azarian whose telephone number is (703) 306-5907 . 

The examiner can normally be reached on Monday through Thursday from 6:00 a.m. to 7:30 

p.m. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 

Bhavesh Mehta, can be reached at (703) 308-5246. 



Contact Information 



Any response to this action should be mailed to: 

Assistant Commissioner for Patents 
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Art Unit: 2625 

Washington, D.C. 20231 
Or faxed to: 

(703) 872-9314, {informal or draft communications, should be clearly labeled to 
expedite delivery to examiner). 

Hand delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington, VA., Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application should be directed to 
T.C. customer service office whose telephone number is (703) 306-0377. 
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Seyed Azarian 
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